System and method for synchronizing databases

ABSTRACT

A data processing method and system including a plurality of databases linked by at least one communication channels, a synchronization set which defines the objects or records to be synchronized between the plurality of databases, and a synchronizer for each database which controls and monitors the synchronization between databases and accesses a local database to which the synchronizer is connected. Each synchronizer includes a communications module which monitors and controls the communication with other databases and at least one table synchronizer which controls and monitors the synchronization of the local database and access to the local database. Each table synchronizer includes a table synchronizer engine which handle the synchronization management for its table synchronizer, a plug-in which handle the generic database communication for its table synchronizer, and a driver which controls communication with the local database. The data processing method synchronizes data records of a source database and a destination database. The method includes defining a synchronization set for data records existing in the source database, synchronizing the source database and the destination database based on the synchronization set, and changing the definition of the synchronization set while synchronizing the source database and the destination database.

FIELD OF THE INVENTION

[0001] The present invention relates generally to a system and method ofupdating data on several databases. More particularly, the presentinvention relates to a system and method of dynamically updatingsynchronized databases connected by a communication channel.

BACKGROUND OF THE INVENTION

[0002] Databases are one of the most widely used tools in computingtoday. A database is a collection of related information about asubject. It is organized in a useful manner that provides a base forprocedures such as retrieving information, drawing conclusions andmaking decisions.

[0003] A database is a collection of objects (called records). Eachobject is made of one or more characteristics (called fields). Similarobjects are often grouped in a list (called table).

[0004] A shared database is a database that can be accessed from manydifferent users residing in different locations. Those locations areconnected through a communication network to the shared database.

[0005] Considering one user accessing to a shared database, the shareddatabase is made of two databases in reality (see FIG. 1). The first oneis the remote database the user want to access; the second one iscontains the local data directly useable by the user's application. Thislocal database often resides in memory and often contains only a subsetof the information contained in the remote database. Conventionally,different models are used to manage those two databases.

[0006] Direct access model: This model only uses the local database asworking memory space. It relies heavily on the communication network tofeed this local memory with the information the user application needs.It fetches all the tables and index data to satisfy the local user'sapplication queries. The direct access model requires all the workingdata (indexes and records) and the result data (records) to betransferred for each query. This is very burdensome on the communicationchannel.

[0007] Client/server model: This model transfers from the server (remotedatabase) to the client (local database) the data corresponding to theresults of the queries, so the user's application can work with thatdata. It relies on the server to execute the queries. There is no localoptimization of the queries. Each query requires a roundtrip from theclient to the server and back to the client.

[0008] Replication model: This model copies the whole database locally.This is a problem because the local database has to keep a copy of allthe data in the remote database whether it is necessary to thesynchronization or not. This model requires the local computer to handlethe full sized remote database. This model only works when bothdatabases have the same format.

[0009] Synchronization model: This model is an extension of thereplication model where the remote and local databases can be ofdifferent formats. Most recent synchronization models allow that only apart of the remote database is transferred locally (synchronizationbased on query). However, current synchronization models cannot beupdated dynamically.

[0010] The direct access and client/server models require fast andpowerful communication network between the client and the server. Theyneed a permanent connection between the remote and the local database.This requires a high availability of the servers because no down time isaccepted by the users. The number of concurrent users is limited by theperformances of the servers.

[0011] The replication model requires heavy administration to keep thedatabases identical. Any error in communication or transaction can breakthis strict synchronization.

[0012] In the replication and synchronization models, thesynchronization definition (the way records are synchronized) can't bechanged, added or removed while the synchronization process is running.Furthermore, if the synchronization definition is changed, the localsynchronized records are not removed accordingly. When a remote recordis changed in such a way that its synchronized status is changed from‘synchronized’ to ‘non-synchronized’, it is not removed from the localdatabase. The replication and synchronization models require the remotedatabase to keep track of all the changes applied to the synchronizedrecords (transaction or update log). This causes heavy processing andslows down operations for applications on the remote side.

[0013] The replication and synchronization models are too static to beused efficiently inside an application working memory, like the directaccess and client/server models are able to do.

[0014] All four models are concerned only with a one-to-onerelationship. There can only be one, well defined remote databaseconnected to one, well-defined local database. Both side of theconnection are dedicated. Communication is done using a specificallydesigned language.

[0015] Furthermore, the user's application cannot access multiple remotedatabases and handle the data as a unique set of records. Each remotedatabase must have one corresponding local database. No data merging ispossible.

[0016] All four models use record level granularity. The smallest dataunit that is transferred is one record (one object).

[0017] Theses and other drawbacks and deficiencies exist in existingsystems and methods.

SUMMARY OF THE INVENTION

[0018] The present invention provides a new way to transfer data. Itallows generic and optimized communication between differently formatteddatabases, proposing an alternative to direct access, client/server,replication and classical synchronization models.

[0019] According to one embodiment of the invention, a data processingmethod and system is provided which includes: a plurality of databaseslinked by at least one communication channel, a synchronization setwhich defines the objects or records to be synchronized between theplurality of databases, and a synchronizer for each database whichcontrols and monitors the synchronization between databases and accessesa local database to which the synchronizer is connected. Eachsynchronizer includes a communications module which monitors andcontrols the communication with other databases and at least one tablesynchronizer which controls and monitors the synchronization of thelocal database and access to the local database and a synchronizerdatabase (SDB) which stores internal synchronization data such as thelast update time of a synchronized record. Each table synchronizerincludes a table synchronizer engine which handle the synchronizationmanagement for its table synchronizer, a plug-in which handles thegeneric database communication for its table synchronizer, and a driverwhich controls communication with the local database.

[0020] The table synchronizers allow any kind of databases to be linked.The database can reside on disk or in memory, they can be of any format,from old ‘flat’ database to recent object oriented database formats (aslong as the database contains a collection of structured objects). Thedatabases do not need to hold specific information aboutsynchronization. The user's application can continue to access thedatabase locally while the communication process with the otherdatabases takes place.

[0021] The synchronizer and the SDB allow the present invention totransfer only needed data from the database and reuse existing data inthe SDB to handle queries. This provides an advantage over the directaccess model.

[0022] The table synchronizer and the SDB interact to combineoverlapping queries with similar requests. By combining queries beforeaccessing the database, the present invention decreases the access timeof the database resulting in local optimization. This provides anadvantage over the client server model.

[0023] The synchronizer and SDB interact to retrieve only the neededdata from the database and store data in the SDB. The retrieval andstorage of only needed or requested data avoid duplication of thedatabase locally. This provides an advantage over the replication model.

[0024] The synchronizer and SDB interact to remove, add, and modifysynchronization definition while still allowing local access to thesynchronized data resulting in dynamic synchronization. Specifically, ifthe synchronization definition is changed, all the components of thesynchronizer that use the information are notified immediately and reactaccordingly, even if they are in the middle of doing something else.This provides an advantage over the conventional synchronization modelsthat deny access to synchronized data while the synchronizationdefinition is being modified, added or removed.

[0025] The synchronizer also allows local synchronized records to beremoved when the synchronization definition is changed. The synchronizerincludes various subcomponents (Qes, Qr, Pr, Pe, Prd, and Ped) thatprocess the synchronization definition change notification. For example,if a synchronization definition is changed, Qes, a synchronizersubcomponent, is notified. Qes then starts a new refresh cycle whichremoves any local records that are no longer part of a synchronizationdefinition. This provides an advantage over conventional systems.

[0026] In one aspect, the data processing method synchronizes datarecords of a source database and a destination database. The methodincludes defining a synchronization set for data records existing in thesource database, synchronizing the source database and the destinationdatabase based on the synchronization set, and changing the definitionof the synchronization set while synchronizing the source database andthe destination database.

[0027] In another aspect, records to transfer are selected by thesynchronization process. This selection allows partial and optimizedcommunication between the databases. The data is transferred with afield level granularity, but databases coherence is respected (allmodified fields for a given object can be transferred together).

[0028] The connection between databases can be discontinuous; thedatabases are updated when the connection is re-established.

[0029] Because of the dynamic nature of the synchronizer, asynchronization definition can be added, changed or removed ‘live’.Synchronized records are added, updated or removed accordingly. Thedefinition can be used to query remote databases, access locally theresulting records and remove those local records when the query resultis not needed anymore.

[0030] Multiple synchronization definitions can provide fields frommultiple remote databases for merging in one record in a local database.The local database contains the net result of the merged informationfrom multiple remote databases. Merging occurs on different levels:records may be merged in a table (table level) or fields in records(record level).

[0031] In another aspect, the present invention provides a system andmethod for synchronizing data records of a plurality of databases. Thesystem and method defines at least one synchronization set of datarecords that exist in one of the plurality of databases (sourcedatabase); synchronizes the source database and one of the plurality ofdatabases (destination database) based on the synchronization set; andallows changes to the definition of the synchronization set while thesynchronization is active or live. The method can remove a data recordfrom the destination database based on the step of changing thedefinition of the synchronization set. The method can also delete a datarecord belonging to the synchronization set from the source database andremove the data record from the destination database based on thedeleting of the data record in the source database.

[0032] In another aspect, the present embodiment provides a system andmethod for synchronizing data records of a plurality of databases. Thesystem and method defines a synchronization set for data recordsexisting in at least one of the plurality of databases; synchronizes theplurality of databases based on the synchronization set; and changes thedefinition of the synchronization set while synchronizing the pluralityof databases. The system and method can remove a data record from atleast one of the plurality of databases based on the step of changingthe definition of the synchronization set. Also, the process ofsynchronizing the plurality of databases can include merging data fromat least two of the plurality of databases into a data record in one ofthe plurality of databases.

[0033] In another aspect, the present embodiment provides a system andmethod synchronizing data records of a source database and a destinationdatabase. The system and method defines a synchronization set for datarecords existing in the source database; synchronizes the sourcedatabase and the destination database based on the synchronization set;and scans the data records in the source database defined in thesynchronization set. The system and method can receive a plurality ofqueries for information on data records defined in the synchronizationset and scanning can include combining the plurality of received queriesbefore scanning the source database.

[0034] In another aspect, the present embodiment of the inventionprovides a system and method for synchronizing data records of a sourcedatabase and a destination database. The system and method defines asynchronization set for data records existing in the source database;synchronizes the source database and the destination database based onthe synchronization set; and creates a notification packet for each datarecord defined in the synchronization set and modified in the sourcedatabase, the notification packet containing only an indication ofmodification. The system and method can define a synchronization set fordata records existing in the source database; synchronize the sourcedatabase and the destination database based on the synchronization set;and create a user flag each data record defined in the synchronizationset and modified in the destination database. Also, the system andmethod can define a synchronization set for data records existing in thesource database; hash information on the data records defined by thesynchronization set; send the hashing information to the destinationdatabase; and synchronize the source database and the destinationdatabase based on the synchronization set. The system and method alsocan receive a request for hashing information on at least on data recorddefined by the synchronization set at the source database from thedestination database and send the at least one data record to thedestination database.

[0035] According to one embodiment of the invention, a computer systemis provided which includes: a processor, a memory coupled to saidprocessor; the memory having stored therein sequences of instructionswhich, when executed by said processor, cause said processor tosynchronize data records of the source database and the destinationdatabase by causing the processor to perform the steps of (a) defining asynchronization set for data records existing in the source database;(b) synchronizing the source database and the destination database basedon the synchronization set; and (c) changing the definition of thesynchronization set while synchronizing the source database and thedestination database.

[0036] According to one embodiment of the invention, an article ofmanufacture is provided which includes a medium readable by a processor,the medium having stored thereon a plurality of sequences ofinstructions, said plurality of sequences of instructions includingsequences of instructions which, when executed by a processor, causesaid processor to perform the steps of: (a) defining a synchronizationset for data records existing in a source database; (b) synchronizingthe source database and a destination database based on thesynchronization set; and (c) changing the definition of thesynchronization set while synchronizing the source database and thedestination database.

[0037] The above and other embodiments, aspects, features, andadvantages of the present invention will become readily apparent fromthe following detailed description that is to be read in conjunctionwith the accompanying drawings.

DESCRIPTION OF THE DRAWINGS

[0038]FIG. 1 illustrates an example of shared databases.

[0039]FIG. 2 illustrates the basic synchronization components accordingto one embodiment of the invention.

[0040]FIG. 3 illustrates atomic synchronization between two databasesaccording to one embodiment of the invention.

[0041]FIG. 4 illustrates a network of atomic synchronization accordingto one embodiment of the invention.

[0042]FIG. 5 illustrates storage of specific synchronization dataaccording to one embodiment of the invention.

[0043]FIG. 6 illustrates synchronizers in a network of synchronizationsaccording to one embodiment of the invention.

[0044]FIG. 7 illustrates a loop of atomic synchronizations according toone embodiment of the invention.

[0045]FIG. 8A illustrates a synchronizer overview according to oneembodiment of the invention.

[0046]FIG. 8B illustrates the content of the SDB according to oneembodiment of the invention.

[0047]FIG. 9 illustrates an initial phase between 2 synchronizersaccording to one embodiment of the invention.

[0048]FIG. 10 illustrates SDB A and B before the start of theinitialization phase according to one embodiment of the invention.

[0049]FIG. 11 illustrates SDB A and B after initialization 1 packetaccording to one embodiment of the invention.

[0050]FIG. 12 illustrates SDB A and B after initialization 2 accordingto one embodiment of the invention.

[0051]FIG. 13 illustrates SDB A and B after reception of twoinitialization packets 1 crossing each other according to one embodimentof the invention.

[0052]FIG. 14 illustrates SDB A and B after reception of twoinitialization packets 2 crossing each other according to one embodimentof the invention.

[0053]FIG. 15 illustrates table synchronizer subcomponents according toone embodiment of the invention.

[0054]FIG. 16A illustrates a Qes loop process according to oneembodiment of the invention.

[0055]FIG. 16B illustrates a Qes loop process according to oneembodiment of the invention.

[0056]FIG. 17 illustrates a filter process flowchart according to oneembodiment of the invention.

[0057]FIG. 18A illustrates a Prd process according to one embodiment ofthe invention.

[0058]FIG. 18B illustrates a Prd process according to one embodiment ofthe invention.

[0059]FIG. 19 illustrates a Ped process according to one embodiment ofthe invention.

[0060]FIG. 20 illustrates a Qr process according to one embodiment ofthe invention.

[0061]FIG. 21 illustrates a Qe process according to one embodiment ofthe invention.

[0062]FIG. 22 illustrates a Pr process according to one embodiment ofthe invention.

[0063]FIG. 23 illustrates a Pe process according to one embodiment ofthe invention.

[0064]FIG. 24A illustrates field matching between database A and B forthe creation of one record in a source database example according to oneembodiment of the invention.

[0065]FIG. 24B illustrates a creation of record in a source databaseexample according to one embodiment of the invention.

[0066]FIG. 25 illustrates a synchronization process for a recordcreation example according to one embodiment of the invention.

[0067]FIG. 26A illustrates field matching between database A and B for afield modification and channel disconnection example according to oneembodiment of the invention.

[0068]FIG. 26B illustrates a field modification example according to oneembodiment of the invention.

[0069]FIG. 27 illustrates a synchronization process for a fieldmodification example according to one embodiment of the invention.

[0070]FIG. 28 illustrates a change of synchronization query exampleaccording to one embodiment of the invention.

[0071]FIG. 29A illustrates field matching between database A and B for achange of query, suppression in destination source, and user flagexample according to one embodiment of the invention.

[0072]FIG. 29B illustrates field matching between database A and B for amerging example according to one embodiment of the invention.

[0073]FIG. 29C illustrates field matching between database C and B for amerging example according to one embodiment of the invention.

[0074]FIG. 29D illustrates a field merging example according to oneembodiment of the invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS OF THE INVENTION

[0075] The following description is presented to enable any personskilled in the art to make and use the invention, and is provided in thecontext of a particular application and its requirements. Variousmodifications to the preferred embodiments will be readily apparent tothose skilled in the art, and the generic principles defined herein maybe applied to other embodiments and applications without departing fromthe spirit and scope of the invention. Moreover, in the followingdescription, numerous details are set forth for the purpose ofexplanation. However, one of ordinary skill in the art would realizethat the invention may be practiced without the use of these specificdetails. In other instances, well-known structures and devices are shownin block diagram form in order not to obscure the description of theinvention with unnecessary detail. Thus, the present invention is notintended to be limited to the embodiment shown, but is to be accordedthe widest scope consistent with the principles and features disclosedherein.

[0076] The present invention encompasses a system and method ofsynchronization. Synchronization is defined as an updating process for aset of objects or records residing in different databases 10 and 12 (seeFIG. 2). The characteristics of the objects are the fields. Thedatabases 10 and 12 communicate via a communication channel 14.Communication channel 14 can be any type of communication channel thatallows communication of data over the channel, such as wirelesstransmission channel or fiber optic network.

[0077] Hereinafter synchronization is defined as a combination of atomicsynchronizations. The atomic synchronization is the basic building blockof the synchronization model of a present embodiment of the invention.

[0078] Atomic synchronization is the simplest case of synchronization.It operates on two sets of records 16 and 18 residing in two databases10 and 12 linked by one communication channel 14 (see FIG. 3).

[0079] An atomic synchronization is unidirectional as it defines asource database 10(A) and a destination database 12(B). Fields 20, 22,and 24 are transferred through the communication channel 14: A change ofa field value in the source database 10 causes a change of the fieldvalue in the destination database 12, but the opposite is not true.

[0080] Changed fields are transferred in ‘packets’ representing logicalrecords in both databases 10 and 12 (objects). A logical record is acollection of related fields. It can correspond to one or more physicalrecords.

[0081] The atomic synchronization is further defined by an access tosource database 10 (A) and an access to destination database 12(B). Thesynchronization and local user's applications can access the databasessimultaneously. User's access to the databases 10 and 12 is generallynever blocked nor restricted due to synchronization.

[0082] The communication channel 14 is established between databases 10and 12 on separate computers or on the same computer and carries fieldvalue and synchronization information.

[0083] The communication channel 14 can carry both values of fields andsynchronization information. Modification time (timestamp) of a fieldvalue is a typical example of synchronization information.

[0084] Several synchronizations between databases 10 and 12 can sharethe same communication channel 14 and database access. Typically, thecommunication channel 14 is implemented over TCP/IP.

[0085] The synchronization minimizes as much as possible the informationexchanged over the communication channel 14: only the requiredmodification information is sent and the data is compressed. Thus, theproblem associated with the replication model, i.e. copy the wholedatabase, can be avoided.

[0086] The communication channel 14 can be discontinuous; thesynchronization is responsible for updating values in destinationdatabase 12(B) after the (re) establishing of the communication channel14.

[0087] Detection

[0088] The synchronization must detect in a database any change of valueof a field participating in a synchronization. A change may be either amodification of the value, a creation of a field or a suppression of afield.

[0089] Basically, three detection methods are used:

[0090] Notification: In this case, the database notifies any change ofany field in the database. The notification is directly handled by thesynchronization. The notification is not implemented in all databasesystems.

[0091] Modification query: If the database is equipped with amodification tag at the record or field level, the modifications arequeried and the modified record are directly handled by thesynchronization.

[0092] Scanning: If notification and modification query are notavailable, the only way to detect a change in the database is to scanthe database for all the records and fields conforming to the definitionof the synchronization query.

[0093] The synchronization chooses the best detection method inreal-time. If the database supports notifications and communication withthe database is continuous, notification is used. In case ofdisconnection with the database, scanning or modification query are usedat least once to restore data integrity, then the synchronizationautomatically switches to notification.

[0094] A source record key and a source synchronization field mask canbe used to identify the record 16 in the source database 10(A)containing the fields to be synchronized. The set of synchronized fieldsin the source database 10 is the source synchronization field mask. Useof the synchronization field mask allows the synchronization method toachieve field level granularity because individual fields can beselected as opposed to entire records.

[0095] The selection of fields in the source database 10 can be dividedin two steps. First, a synchronization query selects records and then asynchronization field mask is applied in each resulting record to obtainthe source fields to synchronize.

[0096] The system and method can also include a destination record keyand a destination synchronization field mask. The destination record keycan be used to identify the record 18 in the destination database 12(B)that will receive the synchronized fields. The set of synchronizedfields in the destination database 12 is the destination synchronizationfield mask.

[0097] The following is an example of field matching between database10(A) and database 12(B). There is a one-to-one matching between fieldsin database 10 and fields in database 12. A one-to-many relationship andcomputed fields is also possible.

[0098] Suppose that data sent by database 10 arrives at database 12. Indatabase 12, the corresponding record is selected according to therecord key. Then, fields can be inserted, modified or removed accordingto the following rule:

[0099] If the selected fields are missing in database 12, they areinserted.

[0100] If the fields are still present in database 12 but not indatabase 10, they are removed.

[0101] If the value in database 12 is older, the field is updated.

[0102] The fact that the field is added or removed in database 12 is notonly dependent on the fact that those fields have been added or removedin database 10, it also depends on the fact that the field issynchronized or not. Any change in the source record key or in thesynchronization definition itself can influence the presence of thefield in the database 12.

[0103] Atomic synchronizations can be combined to provide a network ofsynchronizations between any numbers of databases (see FIG. 4).

[0104] Database 10 ⇄database 12: Bi-directional network ofsynchronizations is achieved by linking 2 databases with 2synchronizations. The atomic synchronization definition from database 10to database 12 can differ from the atomic synchronization definitionfrom database 12 to database 10.

[0105] For example, suppose that the atomic synchronization fromdatabase 10 to database 12 is defined on field 1 and 2 while thesynchronization from database 12 to database 10 is defined only on field1. In this case a change of field 1 in database 10 will cause the updateof field 1 in database 12 and conversely. In opposition, a change onfield 2 in database 10 will force the updated in database 12 but achange on field 2 in database 12 will not force the update in database10.

[0106] Database 10, database 28→database 12: Multiple atomicsynchronizations can feed one database. The data is merged in thedestination database. Records from different source databases willappear together in the same destination database. Furthermore, since thedestination record is selected on a key basis, fields from differentdatabases can be merged into the same record. Therefore there are 2level of data merging: at the table level, where records are merged in acommon table, or at the record level, where fields are merged in acommon record. Database 10→database 12→database 30: Cascadingsynchronizations will provide caching in multiple databases. Data isreplicated in the intermediate database 12.

[0107] Synchronization of atomic synchronizations

[0108] For each database, the synchronization process stores specificsynchronization data in an internal synchronization database (SDB forSynchronization DataBase). Each SDB contains three types of data:

[0109] Field Data is data relative to the synchronized fields of onedatabase. Timestamp or unique identifiers are examples of field data.

[0110] Database Structure Data (or Meta-Data) are data relative to thestructure of database participating to a network of synchronizations(Key definition, fields definition, etc).

[0111] Synchronization Data is data relative to all atomicsynchronization definitions belonging to a network of synchronizations(synchronization query, field mask, destination record key, etc).

[0112] Synchronization Meta Data can be synchronized. In other words,each SDB can know everything about the synchronized table structures andatomic synchronizations belonging to adjacent nodes in the network ofsynchronization.

[0113] It is not necessary to define a synchronization to force thesynchronization of the Meta-Data. Meta-Data of a database issynchronized as soon as the database is attached to a network ofsynchronizations.

[0114] Because the structure of synchronized databases (database 10 anddatabase 12) can be published, all the details of the database can beavailable remotely. This information can be used to define thesynchronization. A synchronization definition can be remotely created,changed and removed. The changes can be propagated in real-time, whilesynchronizations are still running.

[0115] For example, in FIG. 5, it is possible to create from database 12an atomic synchronization from database 12 to database 10 without havingto phone the IT manager of the site A to ask him the structure of thedatabase 10.

[0116] The database structure may be changed. The same method can beused to create remote empty databases (auto-create).

[0117] For example, two machines A and B linked by a communicationchannel 14 are participating in a synchronization network. Database 10is created on machine A, its meta-data appear in database 12. There isan auto-create mechanism creating first an empty database on machine Bwith the same structure as database 10. From this moment, a defaultatomic synchronization is initiated from database 10 to database 12. Anexample of default synchronization could be a synchronization based on aquery selecting all fields in database10 so that the database 12 will bea replication of database 10.

[0118] Optionally, synchronization information can be merged in onecentral database to allow centralized management of synchronizations.Any change to a synchronization definition in the central database isforwarded to the involved synchronizations.

[0119] In one embodiment, in order to support any network of atomicsynchronizations, the present invention links each database to onesynchronizer (see FIG. 6). Each database is preferably only in contactwith one synchronizer. A synchronizer can keep information about thedatabase without any modification to the database structure. This allowsthe system and method to synchronize databases with different formats orstructures.

[0120] There are many advantages for a decentralized architecture ofsynchronization information. The load of the synchronization process canbe spread over several machines. Each synchronizer can be autonomous andnot depend on a central server; it keeps in its SDB (SDB forSynchronization DataBase) up to date information over the DBindependently of the status of communication channel.

[0121] Synchronization

[0122] A synchronizer manages all atomic synchronizations for which itsassociated database is the source or the destination. It is responsiblefor all the synchronization data going to or from a given database.

[0123] Synchronizers talk to each other through the communicationchannels. Once the synchronizer has been started, it is ready tocommunicate with other synchronizers (ports are open) and its localdatabase (opened database). See FIG. 6.

[0124] Inside a synchronizer, the processing is generally based onfields of a given record. Within a synchronizer, records are locallyidentified by a local ID. Within a network of synchronization, recordsare globally identified by a global ID. The same applies for fields.

[0125] The concept of global ID allows closing a loop in a network ofsynchronization without duplication of records. For example, in loopdatabase 10→database 12→database 26 presented in FIG. 7, the record32(X) is present only one time in each synchronizer 34, 36, and 38.Assume that at the beginning a record 32(X) is only present in database10. A global ID 40 of record 32(X) is assigned to record 32 insynchronizer 34. Because of the synchronizations database 10→database 12and database 12→database 26, the same global ID 40 is inserted insynchronizers 36 and 38. In the same way, the record 32 is inserted indatabase 12 and 26. Because of the Global ID 40 which is unique for allthe network of synchronization, the synchronization database 10→database26 will not cause the replication of record 32 in database 10.

[0126] Synchronizer tasks

[0127] The synchronizer manages atomic synchronizations. It isresponsible for outgoing and incoming synchronizations.

[0128] Outgoing synchronizations: The synchronizer gets the modifiedfield information from the database. It updates its internal database(SDB), then, if needed, it sends the fields to the other synchronizers.It handles notifications coming from the database. It handles refreshingof data in SDB in case of disconnection from the database (scanning). Ithandles the scanning requests from other synchronizers.

[0129] Incoming synchronizations: The synchronizer gets the modifiedfield information from the other synchronizers (scanning ornotification). It updates its SDB (including a track of importedfields), then, if needed, it modifies the fields of the database. Ithandles refreshing in case of disconnection from other synchronizers(scan). It handles notifications from other synchronizers. It removesnon-synchronized fields from database.

[0130] The synchronizer, in conjunction with the SDB, uses the storedsynchronization information and reuses local data to allow the presentinvention to transfer only needed data, thus overcoming the heavy burdenon a communication channel associated with the direct access model.

[0131] In one embodiment as shown in FIG. 8A, a synchronizer 42 iscomposed of one communication module 44 and as many table synchronizers46, 48, and 50 as tables present in the database 52 (see FIG. 8A).

[0132] The communication module 44 is the synchronizer interface to thecommunication channel 54. The communication with other synchronizers isdone using the TCP/IP protocol over channel 54.

[0133] For outgoing synchronizations, the module 44 is responsible forrouting the packets to the right synchronizers while for incomingsynchronization, the module 44 routes the packet to the right tablesynchronizer engine.

[0134] Table synchronizers 46, 48, and 50 manage atomic synchronizationson fields presented by the respective drivers 56, 58, and 60 in thelogical tables of the synchronizers 46, 48, and 50. Table synchronizers46, 48, and 50 manage all the atomic synchronization going to or comingfrom the logical tables. Table synchronizers 46, 48, and 50 are eachcomposed of a driver 56, 58, and 60, a plug-in 62, 64, and 66 and atable synchronizer engine (sync. eng.) 68, 70, and 72, respectively. Anymultiple of table synchronizers can be present in one synchronizer.

[0135] Drivers 56, 58, and 60 are the specific database interfaces. Thislayer is preferably as thin as possible and is a variable part of thetable synchronizer because it depends on the type of database it isaccessing. There is one driver for each plug-in. The drivers enable thepresent invention to synchronize databases with different formats orstructures, thus overcoming a problem associated with conventionalmethods.

[0136] Drivers 56, 58, and 60 are also responsible for presenting datafrom the database 52 in a coherent manner. In fact, it presents the datain logical tables so that fields are present in one logical table only.

[0137] Plug-ins 62, 64, and 66 handle the generic database communicationfor its logical table. There is one plug-in per table synchronizer.

[0138] Plug-ins 62, 64, and 66 are the unique accesses to the logicaltable of the synchronizer 42. They have read/write access to thedatabase 52 and the local SDB. Plug-ins 62, 64, and 66 exchange packetswith the local table synchronizer engine. Because all the informationcontained in the packets are in the database format, this information isgeneric.

[0139] Table synchronizer engines 68, 70, and 72 handle thesynchronization management for their respective logical table. There isone table synchronizer engine per table synchronizer.

[0140] Table synchronizer engines 68, 70, and 72 have access to thelocal SDB. They communicate locally with their respective plug-in 62,64, and 66, with other table synchronizer engines running on the samedatabase 52 and with remote table synchronizer engines located on othermachines (via the module 44). From the table synchronizer engine pointof view, the plug-in acts just like a “local” table synchronizer engine.

[0141] The synchronization database (SDB) 73 is storing all theinformation needed by the synchronizer and table synchronizer. It isdivided in three table definitions (see FIG. 8B).

[0142] The first one, the record SDB (Rec-SDB) 78 contains informationabout synchronized records (IDs) of the database and about theirassociated fields (IDs, Timestamps, hashing and user flag). Informationabout fields is multi-valuated: for one SDB field of type “record” (forexample Record Global ID) may correspond to several SDB fields of type“field” (for example, Field ID).

[0143] The second one, the Synchronization SDB (Synchro-SDB) 76 containsthe definition of the atomic synchronization running in a givensynchronizer. This is information about the synchronization itself(Synchro global ID, Source Synchro Global ID, Destination Synchro globalID, Global Ids of local field to be transferred, Global Ids of remotefields to be transferred, Query) and the status of the synchronization(Status, Reset Synchro and initiated Synchro).

[0144] The last one, the table synchronizer SDB (TSyzer-SDB) 74 containsthe definition of the local and adjacent Table Synchronizers (Tablesynchronizer global ID, IP address) but also information about thestructure of the table (meta-data) (Key definition, Field name globalIDs, Field name local IDs) and other information about the databaseTable to synchronize (Record Table Name, Last modification date of DB).

[0145] There is one TSyzer-SDB and one Synchro-SDB per Synchronizer andone Rec-SDB per Table Synchronizer.

[0146] Initialization phase between two synchronizers

[0147] As previously mentioned, synchronization definition and meta-dataare synchronized between adjacent synchronizers. The fields synchronizedas marked with an “S” in FIG. 8B. It means that in each synchronizer, afirst Table synchronizer will be responsible for the synchronization ofTable synchronization definitions and another second Table synchronizerwill be responsible for the synchronization of Table Synchronizersdefinitions. These two Table synchronizers are called “system” TableSynchronizers.

[0148] This paragraph describes how the communication between twosynchronizers is initiated. The global process is presented in FIG. 9.

[0149] In this example, there are two synchronizers A (Server) 80 and B(Client) 82. In each of TSyzer-SDBs 84 and 86, there is one recordrepresenting the Table synchronizer synchronizing the synchronizationdefinitions 88 and 90 and a second one representing the Tablesynchronizer synchronizing the Table synchronizer definitions 92 and 94(see FIG. 10). Those records are called system records. Initially,synchronizer 82 requests a connection to synchronizer 80. The client 82sets a TCP/IP connection 96 with the server 80. The server 80 acceptsthe connection and opens a second connection 98 from server 80 to client82. The client 82 sends a first “init” packet containing Global IDs ofits two Table Synchronizers. The server 80 receives the packet anddecides to establish the communication. If it refuses the connection, itcloses the TCP/IP connection 96.

[0150] The server 80 adds two system records 100 and 102 in itsTSyzer-SDB representing the two remote system Table Synchronizers ofclient 80. It also adds four system records 104, 106, 108, and 110representing the four synchronizations between the two system TableSynchronizers 88 and 92 in server 80 and the two system TableSynchronizers 90 and 94 in client 82 (two in one way, two in the otherway) (see FIG. 11). Those synchronizations are called systemsynchronizations.

[0151] The server 80 sends to the client an init packet 2 with globalIDs of its two system records 100 and 102 representing Tablesynchronizers and global IDs of the four records 104, 106, 108, and 110representing the system synchronizations. The client 82 receives thepacket and adds these six records 112, 114, 116, 118, 120, and 122 inits SDB (see FIG. 12).

[0152] Problems arise if the init packets cross each other. For example,assume that at the same time client 82 sends an init packet 1 ab toserver 80 and server 80 sends an init packet 1 ab to client 82. The FIG.13 presents the situation after the reception of the packets.

[0153] After the reception of the packet 1 ab, client 82 is sending aninit packet 2 ba while server 80 is sending an init packet 2 ab eachcontaining global IDs of its two system records representing Tablesynchronizers and global IDs of the four records representing the systemsynchronizations. To avoid the duplication of the four system records ofsynchronization definition in A and B, the records with global IDsalready present in SDB are replaced by records arriving if the theirglobal ID is inferior to the corresponding record arriving in packet 2.For example, the record “Synchro def A->B” 124 with ID12 in client 82will be replaced by the record “Synchro def A->B” 126 with ID16 comingfrom server 80. The situation after the reception of the two packets 2is presented in FIG. 14.

[0154] In this example, the system records are not synchronized. Ifneeded, they are removed after a timeout.

[0155] Start/Stop of synchronizations

[0156] A synchronization may be active or disable. A synchronization isactive if its destination Table Synchronizers is ready to receive data.If the destination Table Synchronizers is not ready, the atomicsynchronization must not ask its source table synchronizer to send datathrough the communication channel. As soon as the destination TableSynchronizers is ready, it sends a start packet to inform the sourcetable synchronizer that it may sends data concerning thesynchronization.

[0157] Table Synchronizer Subcomponents

[0158] The table synchronizer, through its subcomponents, in conjunctionwith the SDB, allows the present invention to combine overlappingqueries to cut down the amount of access to the local database, thusovercoming a disadvantage associated with conventional synchronizationmodels.

[0159] A table synchronizer 128 with its subcomponents is represented inFIG. 15. There are two main types of subcomponents: Q and P. Qsubcomponents manage Queries made on database 130 and SDB 132 while Psubcomponents manage Packets exchanged during a synchronization process.

[0160] Hereinafter, the Q and P subcomponent are associated with twosubscripts (E or R) indicating if the subcomponent emits (E) or receives(R) data from other subcomponents. Subcomponents of same type withsubscript E (R) emits (receives) data to (from) components withsubscript R (E). For example Ped 134 emits data to Pr 136, Pr 136receives data from remote Pe (Pe of another Sync. Engine).

[0161] Subcomponent are separated into different groups: Qes 138, F 140,Ped 134 and Prd 142 are part of the plug-in 144. They are responsible ofgeneric interaction with the database 130 through the driver 146.

[0162] Qr 148, Qe 150, Pr 136 and Pe 152 are part of the synchronizerengine 154. They provide the link with the other synchronizers and theirdatabase is SDB 132.

[0163] M 156 and SDB 132 are part of the internal database system of thesynchronizer 128.

[0164] Srv 158 represents the Data Base server and Pes 160 thenotification server of the DataBase 130.

[0165] Subcomponents exchange packets between each other. Generally onepacket is sent for information about one record. They do so inside agiven synchronizer or between different synchronizers.

[0166] For the flow of a packet, there are three modes:

[0167] Normal: (the default mode). Arrows in FIG. 15 indicate the flowof normal packets.

[0168] Request: in this mode, packet are going in the opposite directionas for a normal packet.

[0169] Request response is a normal packet but without filtering by Pr136 and dispatching by Pe 152.

[0170] A packet contains information about: Records, Record ID,Synchronization membership, Hashing of timestamps, Fields, Field ID,Value, and Timestamp.

[0171] Flags can be used to indicate the type of packet:

[0172] Request

[0173] Response to a request

[0174] Reset

[0175] End of cycle

[0176] Start

[0177] Stop

[0178] A packet might contain only a subset of this information. Theformats possible for a packet are:

[0179] “Basic”: Record ID+synchronization membership;

[0180] “Hashing”: “Basic” +a hashing calculated on all fieldsparticipating to synchronization (A hashing function is a functioncompressing the information. A hashing is not reversible: it is notpossible to retrieve the initial value from a hashing on this value.);

[0181] “Fields”: “basic” +field Ids;

[0182] “Timestamps”: “Fields” +fields timestamps; and

[0183] “Data”: “Timestamps” +field values.

[0184] They are two mode for Qes 138: the scanning detection methodwhere the goal is to update both the timestamps and the membership tosynchronization and a light scanning detection method where Qes onlycheck for synchronization membership.

[0185] As already mentioned, the scanning method is used both for theinitializing phase and when notification is not available. The start ofa synchronizer or the change of a query when notification is used areevents demanding an initializing phase. Qes 138 loops through the recordonly when needed. That is when notification is inactive (because thedatabase does not support it or because the channel is closed) or whenthe data integrity needs to be restored (after a lost of connection oran error).

[0186] Qes is running in light scanning method is used when timestampsare fields contained in the DB itself. In this case, the implementationis lighter because, Qes 138 does not need to estimate the timestamps.

[0187] Inside a Synchronizer Table, the main role of Qes 138 is toestimate timestamps associated with fields that have been modified inthe DB 130. Qes 138 is based on a loop considering all queries definedin a table synchronizer. From the resulting records, it selects allfields participating to atomic synchronizations. Qes 138 analyzes eachrelevant field separately.

[0188] Qes In/out

[0189] In:

[0190] From Srv 158, Internal Key value (if relevant)+timestamps (ifavailable) or Internal Key value (if relevant)+Fields value. An internalkey is a key generated by the DB 130.

[0191] Out:

[0192] to Qr 148, Basic packet with flag reset

[0193] to Qr 148, Basic packet with flag end of cycle

[0194] to Qr 148, data packet

[0195] The steps of the Qes 138 algorithm are (see FIGS. 16A and 16B):

[0196] Step 162, a “Qesloop” flag is attached to each synchronization.Step 164, the Qesloop flag is set to false for the set S ofsynchronizations attached to a given table synchronizer.

[0197] Step 166, determine if at least one synchronization has the flagto false. If at least one synchronization has the flag set to false,proceed to step 168, otherwise proceed to step 170 and stop.

[0198] Step 168, get the first synchronization S1 in S with flag tofalse and read in SDB 132 the Query Q associated with S1.

[0199] SQ is the set of atomic synchronizations having the same query Qas synchronization query.

[0200] Step 172, read in SDB 132 the set SQ of synchronizations withQuery Q as query and put all flags to true for those synchronizations.

[0201] Step 174, send a basic packet with flag reset to Qr 148 toindicate the start of the treatment of a query.

[0202] Step 176, if there are still records in DB 130 found with queryQ, proceed to step 178, otherwise proceed to step 180.

[0203] Step 178, Qes creates a packet.

[0204] Step 182, read next record in DB 130.

[0205] Step 183, if record does not exist yet in SDB 132, create it inpacket to Qr, put membership to true for source synchronizations of SQ.For a record, a membership to a given synchronization set to trueindicates that this record is participating to the synchronization.

[0206] Step 184, if Qes is in mode “notification with timestamp managedby DB” proceed to step 186.

[0207] Step 185, treat each record fields of the synchronization fieldmask. If there is still records to treat, proceed to step 188, otherwiseto step 186.

[0208] Step 186, send the packet to Qr 148.

[0209] Step 188 consider the next field to treat. Add field ID in packetto Qr 148.

[0210] Step 190, determine if timestamps are present in the packet. Ifyes, proceed to step 192, otherwise proceed to step 194. The presence oftimestamp depends on the capacity of DB 130 to generate timestamps.

[0211] Step 192, if timestamp received from DB 130 is more recent thanthe timestamp in SDB 132, put the timestamp received from DB 130 inpacket. Otherwise, put a predefined value “OldestValue” in the packet.The Oldest Value is the oldest value known by the synchronizationsystem.

[0212] If needed Update hashing function in SDB 132 and the user flag(In fact, the writing in SDB 132 does not have to occur immediately. Qes138 can wait for an acknowledgement (more precisely a return onfunction) confirming that the packet has been correctly sent by Pr 136.)

[0213] Step 194, if timestamps are not available, calculate a hashingvalue on the value received from the DB 130 and compare it with thehashing value in SDB 132.

[0214] If the hashings are different, Qes 138 estimates (Qes 138 basesits estimation on the fact that the modification has occurred sometimesbetween the start of the previous Qes 138 loop and the current time) thetimestamp and puts it in the packet to Qr 148. If the hashings are thesame, it sends the pre-defined “OldestValue” in the packet to Qr 148. Ifneeded Qes 138 updates hashing function in SDB 132 and the user flag (Infact, the writing in SDB 132 does not have to occur immediately. Qes 138can wait for an acknowledgement (more precisely a return on function)confirming that the packet has been correctly sent by Pr 136.)

[0215] Step 186, when all the field of a given record have been treated,send the packet to Qr.

[0216] Step 180, when all the synchronizations of SQ have beenconsidered, send a end of cycle packet to Qr

[0217] A filter (F) 140 receives a notification for any modification,creation or suppression. This notification is generated directly by thedatabase 130 or by a query made on modification tags in the databaserecords.

[0218] The filter 140 checks the field membership to any atomicsynchronization in SDB 132. Filter 140 generates and sends packet to Pr136 containing only information for fields participating in one atomicsynchronization at least.

[0219] A packet sent by F 140 to Pr 136 contains: field value, fieldtimestamp (equal to current synchronization time) and synchronizationmembership.

[0220] The filter 140 sets the timestamps in the packet.

[0221] If needed, F 140 updates the hashing value in the SDB 132.

[0222] Filter In/Out

[0223] In from Srv: 158, a record with all field values and the list ofmodified fields (timestamps=current time). The internal key (ifrelevant)

[0224] Out to PR 136, a packet with modified or inserted field valuesand timestamps and synchronization memberships.

[0225] Filter

[0226] Filter 140 algorithm is shown in FIG. 17.

[0227] Step 202, create a packet. This is the packet in which thechanged fields of the current record will be added. Create a emptyvector S of Synchro Global ID. This vector will contain all the SynchroGlobal IDs to which the record belongs. Use in step 234

[0228] The first loop 198 describes outgoing synchronizations. Outgoingsynchronizations are synchronizations for which the current table is thesource. This first loop 198 determines to which outgoingsynchronizations the record belongs.

[0229] Step 204, select first synchro for which the current table is thesource. Step 206, compute the record membership of the selected synchrofrom the values in the record Step 208, determine if the record belongsto the synchronization by determining if the record is selected by thedefinition query of the synchronization. If it does, proceed to step210, otherwise proceed to step 212.

[0230] In step 212, notification of a modification is made by adding atag of belonging to the synchronization in the packet. Add the SynchroGlobal ID to S. If the record does not exist in the SDB record table ofthe Table Synchronizer, get a new Record Global ID and create a entry inthe SDB record table for the new record. Otherwise, if the record wouldbelong to the synchronization in the SDB record table, add a tag ofsuppression to the synchronization in the packet and add the SynchroGlobal ID in S, step 216. Otherwise, it has already been eliminated orhas never belonged to the synchronization.

[0231] Second Loop 220 describes incoming synchronizations.

[0232] Steps 222, 226 and 228—Loop implementation. For eachsynchronization for which the table is the destination:

[0233] Step 224, calculate the belonging to the synchronization from thevalues in the record. If the record belongs to the synchronization, addthe Synchro Global ID in S.

[0234] Step 230, if S is not empty (That means there exists a tag ofbelonging or suppression in the packet or if the record belongs to atleast one incoming synchronization), proceed to step 234.

[0235] Step 234, add the Record Global ID to the record in the packet.

[0236] In step 234, for each, add the Synchro Global ID in S.

[0237] Step 234, for each field belonging to the currentsynchronization: if not already in the packet and if the timestamp inthe record is more recent than the timestamp in the SDB, find the FieldName Global ID from the File Name Local ID, add the field value, theField Name Global ID and the timestamp of the packet, and if this is notalready the case, put the flag user to true in the SDB 132 record tableof the fields.

[0238] Step 236 Send the packet to PR 136.

[0239] Step 238, PRd 142 writes (creates or deletes) fields in DB 130 onrequest of Pe 152 and updates hashing values in the SDB 132 accordinglyif the write operation in the DB 130 was successful. The SDB 132 mustperfectly reflect the state of DB 130. Thus, if the writing in DB 130fails, the SDB should not be updated. To find the destination record,Prd 142 reads in SDB 132 a definition of the key record destination.

[0240] Prd 142 handles two types of request, in and out. For in requestsfrom Pe 152, a packet with modified field values with their timestampsis received. For out requests to Srv 158, a record with modified fieldsand their timestamp (if relevant), and the identification key is sent.

[0241] Prd

[0242] Prd 142 algorithm is shown in FIGS. 18A and 18B.

[0243] Step 240, a packet coming from the Pe 152 containing a RecordGlobal ID (zero =new record in the SDB 132) and the field values withtheir Field Name Global ID and their timestamp as well as the membershipof synchronizations is received by Prd 142.

[0244] Step 241, create a new record=newRecord. For each field in thestructure: convert the Field Name Global ID into Field Name Local ID,convert the Field Name Local ID into an index from the view (which isdetermines the structure of the record), and place the value in therecord at the position pointed out by the index.

[0245] Steps 242 to 256, involve calculation of the key.

[0246] Instance flag key Usable

[0247] Step 242, if Record Global ID=zero (

=>record not yet in SDB), proceed to step 250.

[0248] Step 250, if we are not working with the internal key, proceed tostep 256.

[0249] Step 256, build a value of key from the record.

[0250] Step 254, Key Usable←false

[0251] Step 242, if record is in the SDB, proceed to step 244.

[0252] Step 244, read the value of the key in SDB.

[0253] Step 246, KeyUsable←true.

[0254] Step 294, if we are not working with the internal key, proceed tostep 296.

[0255] Step 296, build a value of key from the record.

[0256] Step 298, if the read key and the built key are not the same,proceed step 300

[0257] Step 300, splitting: remove in the SDB record all belongings tothe synchros referenced in the pack. Set the recordIDw to zero in thepack to force the re-entrance in Prd as a new record.

[0258] Step 302, return true.

[0259] In step 258, identification of the synchronizations and of thetype of operation (Insertion, modification, suppression): if all thesynchronization tags are belonging tags, it is about amodification/insertion.

[0260] Otherwise, if all the synchronization tags are suppression tags,it is about a suppression.

[0261] Otherwise, there is a failure and a Return false is generated.

[0262] Step 258, if modification/insertion, (do not apply modificationif suppression, because modification and suppression separated at thekernel level), proceed to step 260.

[0263] Step 260, if keyUsable=true, proceed to step 268.

[0264] In step 264, create a new record=oldRecord.

[0265] Step 264, read the record in the DB from the key (coming from theSDB) and place the outcome into oldRecord.

[0266] Step 266, for each field in the view: if the field contains avalue in the new record, calculate the value of hashing of the field inoldRecord. If different, then of the value in the SDB (modification inthe DB not yet reflected in the SDB via F or QES) or if the date ofmodification into the oldRecord more recent than the newRecord, deletethe field in the newRecord.

[0267] Step 268, if the record is new in the SDB, proceed to step 272.

[0268] Step 272, if not working with the internet key and update of therecord from the value of the key and the new record failed, insert a newrecord from the value of the key of the newRecord.

[0269] Step 274, if keyusable=false, proceed to step 284.

[0270] Step 276, if use of the internal key, proceed to step 278.

[0271] Step 278, place the received value after writing in the value ofthe key.

[0272] Step 280, rebuild the value of the key from the newRecord ofwhich the values have may be been truncated during the writing.

[0273] Step 282, update the record from the key value of the newRecord.

[0274] Step 284, if the update or insertion of the record has been done,proceed to step 286.

[0275] Step 286, if the file is new in the SDB Record Table, proceed tostep 288.

[0276] Step 288, add a new entry in the record table and place there theRecord Global ID and the key value.

[0277] Step 289, Merging: add in the SDB record the synchro belongingsthat are present in the pack but not yet in the SDB record.

[0278] Step 290, if there are no timestamps in the DB 130, proceed tostep 292.

[0279] Step 292, for every field received: if the entry for this fielddoes not exist in the SDB record, create a new entry. Otherwise, if thehashings are different, update the fields table.

[0280] Step 262, delete the record from the key value and add the RecordGlobal 1D to the packet.

[0281] Step 270, return true.

[0282] Ped

[0283] Ped 134 reads fields in the DB 130.

[0284] Ped 134 updates the hashing function (if needed and if theassociated packet has been successfully passed to the communicationchannel).

[0285] Ped 134 receives packets in from Pr 136. The packet contains alist of request fields (without values). Ped 134 sends out to Pr 136 apacket with the request values read in the DB 130.

[0286] The steps of the Ped 134 algorithm are shown in FIG. 19.

[0287] Step 295, Ped 134 receives a packet from Pr 136 containing theRecord Global ID, the Field Name Local IDs and the Synchro Global ID.

[0288] Step 297, Ped 134 searches in the SDB 132 record table of theplug in 144 for the key of the record from the Record Global ID.

[0289] For every Field Name Global ID

[0290] In step 299, search the Field Name Local ID corresponding in theSDB 132 synchronizers table. Ped 134 is making a request to the DB 130with this key for these Field Name Local 1D. Ped 134 receives a view ofa record coming from the DB 130. This view contains the key and a set ofthe fields with their ID (Field Name Local ID).

[0291] For every field:

[0292] Step 301, add the value of the record in the packet, stamp thepacket nonfilterable, and pass the structure at PR 136. This stampindicates to Pr that the packet is a response to a request. Qr 148 worksonly with Qes 138 and a remote Qe. This is the receiver component forall loops scanning all records in a DB 130 or in the SDB 132. Its mainpurpose is to update in the local SDB 132 the records belonging tosynchronizations.

[0293] The main difference between Qes 138-Qr 148 and remote Qe-Qr 148communications is that the second is remote, and thus remote Qe sendsmore compact packets (hashing format without values) but if a record hasbeen changed, Qr 148 sense a request and will receive the answer withdates.

[0294] Qr

[0295] Qr 148 receives in from remote Qe four types of packets:

[0296] basic format with reset flag;

[0297] basic format with “end of cycle” flag;

[0298] hashing format; and

[0299] date format.

[0300] Qr 148 receives in from Qes 138 packets in a data format(unchanged fields have a coded date). Qr 148 sends out packets to remoteQe in date format with request flag. Qr 148 sends out packets to Pr 136in data format or date format.

[0301] Qr 148 algorithm is shown in FIG. 20. For this example, let P bethe incoming packet which content can be one among those describedabove. Step 304, the Qr 148 process starts when it receives P. Step 306,determine the format of packet P. If the packet P is in hashing format,proceed to step 308. If the packet P is in basic format, proceed to step310. If the packet P is in date or data format, proceed to step 312.

[0302] Step 312, Qr 148 sends packet to Pr 136 and ends process in step314.

[0303] Step 310, Qr 148 searches the first record R in the records tablein the SDB 132 and all refresh levels in R corresponding to eachsynchronization of the packet. Step 316, determine if R exists in SDB132. If not, proceed to step 314 and end process. If R exists, proceedto step 318.

[0304] Step 318, determine if the packet flag is “reset.” If not (flagis “end of cycle”), proceed to step 320. If packet flag is “reset,”proceed to step 322. Step 320, decrement RL by one if it is at two orthree and proceed to step 324 (Note: RL 2 is just an intermediate level.Each record that does not belong anymore to any synchronization is notsend to Qr 148. This record's RL is still 2 before Qr 148 receives the“End of cycle” packet, and it will reach 1 after processing. RL≦1 meansthat the record doesn't belong anymore to a synchronization.) Step 322,set RL to two if it is at three and proceed to step 324. Step 324,search for next R in SDB 132 and proceed to step 316.

[0305] In step 308, Qr 148 searches SDB 132 for R in the record table ofthe record R in the packet. Step 326, determine if R exists in therecord table of SDB 132. If not, proceed to step. If it does exists,proceed to step 330. In step 328, Qr 148 creates a new packet P2 (“date”format without flag) with the same fields as P with all dates set to 0,sends P2 to Pr 136, and ends the process in step 314.

[0306] In step 330, if RL equals two, RL is set to three and proceed tostep 332.

[0307] Step 332, Generate a hashing H with the timestamps in R of allthe fields corresponding to those related in P.Step 334, if H=thehashing in the packet, proceed to step 336. Otherwise, proceed to step338. Step 336, Qr 148 creates a new packet P2 (“request” flag, and“date” format) with the same fields as P, sends P2 to remote Qe, andends process in step 314. Step 338, RL is set to 3 and the process endsin step 314.

[0308] Qe 150 will loop through all the records of the SDB 132 that aresynchronized (outgoing) and send the timestamps to a remote Qr (Qr'). Tominimize communication, it sends a hashed value of the time stamps ofall the synchronized fields. However, on request of Qr', it can send apacket with individual timestamp of fields.

[0309] Qe 150 receives from remote Qr a field packet with “request”flag. Qe 150 sends to remote Qr a hashing packet or timestamps packet orbasic packet with “reset” flag or basic packet with flag “end of cycle.”

[0310] The Qe 150 process is shown in FIG. 21.

[0311] Step 340, start of the process of Qe 150.

[0312] Step 341, if Qe 150 receives from remote Qr field packet with“request” flag, proceed to step 342 If Qe 150 does not receive packet,proceed to step 346.

[0313] Step 342, L is the list of synchronizations stored in SDB 132 forwhich the records of the packet are members.

[0314] Step 343, Qe 150 reads in SDB 132 all the timestamps of all thefield associated with L and sends insert them in a timestamps packetwithout flag

[0315] Step 344 Qe 150 sends the timestamp packet to remote Qr.

[0316] Step 346, Qe 150 initiates the loop and sends to each destinationtable synchronizer a basic packet with “reset” flag.

[0317] Step 347, Qe 150 gets a list of all records for its TableSynchronizer. For all records in the list, Qe 150 performs the followingprocess.

[0318] Step 349, Qe 150 determines if all records in the list have beentreated. If not, Qe 150 gets next record R in list and proceed to step351. If yes, Qe proceeds to step 350. Step 350, Qe 150 sends to eachdestination Table Synchronizer a packet with “end of cycle” flag andends to the process in step 352.

[0319] Step 349, Qe 150 gets from record R the list of synchronizationfor which the membership is put at true in R and put in List Ls ofsynchronizations only outgoing active synchronizations to be reset withRL>=2.

[0320] Step 354, Qe 150 sets Llist to empty. Step 356, Qe 150 determinesif Ls is empty. If yes, proceed to step 349. If no, proceed to step 358.Step 358, Qe 150 adds first synchronization Sfirst in Ls to Llist. Step360, Qe 150 adds to Llist all synchronizations for which TableDestination Synchronizers has the same IP address as the TableDestination Synchronizer of Sfirst. Step 362, Qe 150 calculates ahashing on all fields timestamps participating to one synchronization atleast, puts this hashing in a hashing packet, sets membership to thesynchronization corresponding to Llist, and sends hashing packet toremote Qr with Llist as membership. Step 364, Qe 150 removes Llist fromLs and return to step 356.

[0321] Pr

[0322] Pr 136 centralizes the reception of the packet in thesynchronizer 128 and resolves the conflicts. Conflicts appear when thesynchronization must apply a modification on a field that has beenmodified by the user since last synchronization. Conflict resolution isa method to decide which data will remain on both sides. In oneembodiment, the synchronization model solves the conflict by keeping inthe packet field a more recent value. All field information in theincoming packet that is older than local one is removed.

[0323] Pr 136 updates timestamps and synchronization memberships in theSDB 132 and can send request packets for the missing values.

[0324] Pr 136 receives from Qr 148 packets in date or data format. Pr136 receives from Ped 134 packets in data format with “request” flag. Pr136 receives from Pe 152 packets in data format with “request” flag. Pr136 receives from F 140 packets in data format. Pr 136 receives from aremote Pe packets in data format with “request” flag.

[0325] Pr 136 sends to Ped 134 packets in data format with “request”flag. Pr 136 sends to Pe 152 packets in data format (with or without“response” flag). Pr 136 sends to a remote Pe packets in date formatwith “request” flag.

[0326] The process performed by Pr 136 is shown in FIG. 22. In thisexample, P is an incoming packet and R is the record in the record tableof the SDB 132 associated with the packet. Step 366, the Pr 136 processis initiated by receipt of an incoming packet. Step 368, Pr 136 searchesthe record table in SDB 132 for the record R associated with the packet.

[0327] Step 370, Pr 136 determines flag in packet P. If the flag is“answer to a request,” proceed to step 372. If the flag is “request,”proceed to step 374. If there is no flag, proceed to step 376. Step 372,Pr 136 sends packet to Pe 152 and end process in step 378.

[0328] Step 374, Pr 136 determines if R exists in the record table ofthe SDB (REC-SDB)78. If no, processes is ended in step 378. If yes,proceed to step 380. Step 380, Pr 136 removes from P all wrongmemberships (RL in the record table of the SDB 132 is the reference) andremoves from P all fields that are not transferred by thesynchronizations of P. Step 382, Pr 136 sends P to Ped 134 and endsprocess in step 378.

[0329] Step 376, Pr generates two list of fields ID by using informationfrom R and P:

[0330] FLc: list of fields ID that value is required because they haveno value in P and their timestamp in P is more recent than the timestampof the corresponding fields in R.

[0331] FLv: list of the fields ID that will be put in the packet sent toPe because they have a value in the timestamp of these fields in P ismore recent than the timestamp of the corresponding field R.

[0332] Note:

[0333] FL v and FLc are exclusive (a field cannot belong to both lists).

[0334] If R doesn't exist in the table record, the timestamp of anyfields in R is 0.

[0335] Particular case: if R doesn't exist in the table record and ifthere is no value in P for the fields that make part of the key in thelocal database then FLv is empty and FLc is the list of all the fieldsID in P.

[0336] In step 384, determine if FLv is not empty or R enters in a newsynchronization or R leaves an outgoing synchronization. If thiscondition is fulfilled process to step 386 otherwise process to step388.Note : to determine if R is leaving or entering a synchronization,Pr 136 compares the belonging in the packet with the correspondingrefresh level in R.

[0337] Example: for synchronization S1, if belonging is set at “1” inthe packet and its RL is 0 in R, it means that R enter in S1.

[0338] Step 386, create a new “data” packet P2 with the fields of FLvand add membership “1” for all synchronization of P except for thesynchronizations of SL-which are set to “−1”.

[0339] Note: membership “1” for an incoming synchronization means recorddeletion for the Plug-In 144. The sub-component that decides to remove arecord from the DB 130 is the cleaner (N). This is the reason why, Pr136 transform the incoming membership from “−1” to “1”.

[0340] Pr 136 sends P2 to Pe.

[0341] Step 390 determine if sending is Ok, if it is proceed to step394. If not, proceed to step 392.

[0342] Step 392, Pr 136 deactivates and resets synchronizations of P andend process in step 378.

[0343] Step 394, Pr 136 updates the timestamp of the fields in R withthe timestamp in P of the fields of FLv and set Refresh to true.

[0344] Step 388, Pr 136 determines if FLc is not empty. If not empty,proceed to step 398. If empty, proceed to step 396. Step 396, Pr 136sets Refresh=false and creates a new “request” packet P2 with the fieldsof FLc and the all membership “1” of packet P. In step 396, depending onthe source synchronizer of the synchronizations of P, Pr 136 will sendP2 to Ped 134 or a remote Pe.

[0345] Step 398, If Refresh=true, Pr 136 updates the RL of R accordingto the synchronization membership in P and ends the process in step 378.Otherwise, the process is ended in step 378.

[0346] Pe

[0347] The main role of Pe 152 is to dispatch packets. If necessary, Pe152 reads the synchronization membership in the packet or in SDB 132. Pe152 forwards packets to the right destination table synchronizer(s) (orto the plug-in).

[0348] Pe 152 sends “out of synchronization” packet to a remote Pr (Pr')in order to inform him that a field does not belong anymore to asynchronization (for example, after a modification of thesynchronization query).

[0349] Pe 152 receives from Pr data packets. Pe 152 receives from aremote Pr field packets with “request” flag or basic packet with “reset”flag

[0350] Pe 152 sends to a Remote Pr data packets. Pe 152 sends to Prd 142data packets and Pe 152 sends to Pr 136 field data with “request” flag.

[0351] The Pe 152 process consists of the following steps (see FIG. 23):

[0352] A packet P arrives in step 402. If the packet has a “request”flag (it means that the packet is coming from remote Pr), then forwardit to Pr 136. If a data packet has a “response” flag, Pe 152 removes theflag and sends this packet to the remote destination Table Synchronizeraccording to the synchronization membership of the packet.

[0353] If the format is a data packet containing information about arecord R, Pe 152 puts all synchronizations mentioned in the membershipof the packet into a list L, step 404. In step 404, Pe 152 selects inthe SDB 132 the outgoing synchronizations for which the record R is amember and adds these synchronizations to L if they were not alreadypresent in it.

[0354] Step 406, Pe 152 sets the variable okPlugIn to true. Step 408, Pe152 determines if there is at least one synchronization in L (incomingsynchronization). If yes, proceed to step 410. If no, proceed to step412.

[0355] Step 410, Pe 152 creates a new packet P1 and copies from P to P1the field information associated with the synchronization in L. Step414, Pe 152 sends P1 to plug-in 144. If the send operation failed, Pe152 sets okplugln to false. Step 416, Pe 152 removes the incomingsynchronization from L.

[0356] Step 412, Pe 152 determines if okPlugIn is true. If no, proceedto step 418 and stop the process. If yes, proceed to step 420.

[0357] Step 420, Pe 152 adds to L all outgoing synchronizations definedin SDB 132 for which R is a member. Step 422, Pe 152 removes and resetsfrom L disabled synchronizations.

[0358] Step 424, Pe 152 determines if L is empty. If yes, proceed tostep 426 and end the process. If L is not empty, proceed to step 428.Step 428, Pe 152 gets last synchro Slast in L and adds it to a new listL2. Step 430, Pe 152 adds in L2 all the synchronization of L which havethe same Destination Table Synchronizer. Step 432, Pe 152 creates a newpacket P2, copies from P to P2 the fields information associated withthe synchronization in L, and inserts the synchronization membership(L2). Step 434, Pe 152 sets L=L −L2. Step 436, Pe 152 asks acommunications module to send P2

[0359] Table Synchronizer Examples

[0360] The following section illustrates examples of how each TableSynchronizer subcomponents works during a synchronization process.

[0361] The notation “Qes(a)” means that the subcomponent Qes belongs totable Synchronizer a.

[0362] Creation of one record in a source database

[0363] In this example, two empty databases 438(A) and 440(B) aresynchronized with a synchronization 442(S1) defined as follows (see FIG.24B):

[0364] Source database 438(A) and destination database 440(B)

[0365] Synchronization query SQ1: select [City]=San Francisco

[0366] Source synchronization field mask: [Last Name, First Name, City,Phone]

[0367] Source Record key for selection of records in 438: Key=[LastName, First Name, City]

[0368] Destination synchronization field mask: [Last Name, First Name,City, Phone]

[0369] Record key for selection of records in 440: Key=[Last Name, FirstName, City]

[0370] Field matching between A and B is shown in FIG. 24A. We assumethat database 438(A) does not generate notifications so that thescanning detection method will be used.

[0371] Suppose that empty tables with the same structure already existin 438 and 440, but that a new record 444(A1) is inserted in database438 (see FIG. 24B).

[0372] The synchronization steps are (see also FIG. 25):

[0373] The database 438 notifies F(a) 446 that a record 444(A1) has beenadded. F(a) 446 reads the field values in database 438.

[0374] F(a) 446 reads all the synchronization queries defined in SDB(a)448 in order to find the synchronization(s) the record belongs to. Inthis case, F(a) 446 finds that the record 444(A1) belongs tosynchronization 442(S1). Indeed, the record 444(A1) responds to thesynchronization query {SQ1: select [city]=San Francisco}.

[0375] F(a) 446 creates the corresponding fields and inserts in theSDB(a) 448 a hashing function of the field values (A default value oftimestamp is also inserted at the creation time. This value is the“OldestValue” timestamp” known by the synchronization system). In fact,the writing in the SDB(a) 448 does not occur immediately. The processwaits for an acknowledgement (more precisely a return on function)confirming that the packet has been sent from Sa 450 to Sb 452 via thecommunication channel 454.

[0376] F(a) applies the source field mask, creates and sends a packet 1to Pr(a) 456 with the 4 field values, the 4 timestamps (current time)and their associated synchronizations (S1 in this case).

[0377] Pr(a) compares for each field in packet 1 the timestamps in thepacket with the timestamps of the SDB(a). Because Pr(a) detects adifference for all fields Pr(a) does not remove any information andforwards the packet 1 to Pe(a) 458.

[0378] Pe(a) reads the Synchronization definition of S1 in SDB(a). Fromthis, Pe(a) knows the destination database and the communication channelto use. Pe(a) sends to Pr(b) 460 the packet 1.

[0379] Pr(b) receives packet 1. It compares the timestamp of fields inpacket 1 with the corresponding timestamp in SDB(b) 462 and detects acreation of record. It creates the associated fields in SDB(b) andinserts the timestamps in SDB(b). In fact, the writing in SDB(b) doesnot occur immediately. The process waits for an acknowledgement (moreprecisely a return on function) confirming that the field value has beeneffectively written in the database 440.

[0380] Pr(b) sends packet 1 to Pe(b) 464.

[0381] Pe(b) detects that the synchronization is a incomingsynchronization and sends the packet 1 to Prd(b) 466.

[0382] Prd(b) creates the record in database 440 and writes the value ofthe 4 fields contained in the packet 1 into the corresponding fields ofdatabase 440. Finally, Ped(b) 468 updates the hashing function of the 4fields in SDB(b).

[0383] Field modification & Channel Disconnection

[0384] This example illustrates the synchronization process after afield modification in a source record. The modification occurs when thechannel communication is down.

[0385] In this example, two databases A 470 and B 472 situated in SanFrancisco and Brussels respectively and containing one table (see FIG.26B). The synchronization 474(S1) is defined by:

[0386] Source database 470(A) and destination database 472(B)

[0387] Synchronization query SQ1: select [city]=San Francisco

[0388] Source synchronization field mask: [Last Name, First Name, City,Phone] Source Record key for selection of records in A: Key=[Last Name,First Name, City]

[0389] Destination synchronization field mask: [Last Name, First Name,City, Phone]

[0390] Record key for selection of records in 472: Key=[Last Name, FirstName, City]

[0391] Field matching between 470 and 472 is shown in FIG. 26A.

[0392] In this example, the database 470 is not able to generatenotification. The detection method for changes in database 470 will bethe scanning method.

[0393] A modification of the phone number in the record 476(A1) in thesource database 470 takes place, the phone number changes from (1 415931 1475) to (1 415 842 5641).

[0394] The synchronization steps are (see FIG. 27):

[0395] Qes(a) 478 launches the synchronization query SQ1 {selectcity=San Francisco} on database 470. Srv(a) 480 returns the record476(A1). Qes(a) selects fields by applying the field mask [Last Name,First Name, City, Phone] corresponding to 474 (S1).

[0396] Qes(a) 478 calculates a hashing function on each field of themask and compares the result with the hashing present in the SDB(a) 482.For the fields [Last Name, First Name, City], Qes 478 associates thetimestamp 0 because it detects with the comparison of hashing valuesthat no modification occurred. Otherwise, Qes 478 detects a modificationfor the field [Phone] and estimates the timestamp. Qes 478 based itsestimation on the fact that the modification has occurred sometimesbetween the start of the previous Qes 478 loop and the current time.

[0397] Qes(a) sends a packet 1 with the timestamps and the field valueto Qr(a) 484 and the associated synchronization S1.

[0398] Qes 478 updates the hashing in SDB(a). In fact, the writing inSDB(a) does not occur immediately. The process waits for anacknowledgement (more precisely a return on function) confirming thatthe packet has been sent from Sa 486 to Sb 488 via the communicationchannel 490.

[0399] Qr(a) just forwards the packet 1 to Pr(a) 492.

[0400] Pr(a) compares for each fields in packet 1 the timestamps in thepacket with the timestamps of the SDB 482. Pr(a) detects a differencefor the field [Phone] and updates this timestamp in SDB 482. In fact,the writing in SDB(a) does not occur immediately. The process waits foran acknowledgement (more precisely a return on function) confirming thatthe packet has been sent from Sa to Sb via the communication channel.

[0401] Pr 492 filters this packet 1 removing from the packet informationrelative to non-modified fields [Last Name, First Name, City]. Onlyinformation relative to the field[Phone] remains in packet 1.

[0402] Pe(a) 494 cannot send the packet 1 to Sb since the communicationchannel is interrupted.

[0403] Suppose that the communication channel between 470 and 472 isrestored.

[0404] Qe(a) 496 detects that the communication channel is up. Qe(a)scans SDB(a) synchronization by synchronization. It first sends a packetto Qr(b) 498 to advertise that the following packets concern S1 and thenprocesses each records of S1. Because S1 concerns one record (A1), Qe(a)sends only one packet 2 to Qr(b) containing a hashing functioncalculated on all timestamps in SDB(a) of fields in the field mask ofS1. Qe(a) sends a packet to inform Qr(b) that the processing on S1 isfinished.

[0405] Qr(b) receives the packet 2 from Qe(a) and calculates the samehashing function but this time on field timestamps in SDB(b) 500. Itcompares this hashing with the hashing in packet 2 and deduces that amodification occurred for one of the field of the field mask of S1.Qr(b) sends a request packet 3 to Qe(a) to ask timestamps of each fieldof the field mask.

[0406] Qe(a) answers with packet 4 with timestamps for the for fields ofrecord A1 in SDB(a).

[0407] Qr(b) forwards the packet 4 to Pr(b) 502.

[0408] Pr(b) receives the packet 4 and compares the timestamps in thepacket with timestamps in SDB(b). Timestamps are different only for thefield [Phone]. Pr(b) sends a request packet 5 to Pe(a) in order to getthe value for the field[Phone].

[0409] The packet 5 passes through Pe(a), Pr(a) and arrives to Ped(a)504, which extracts the value of Field[Phone] in record A1 of database470.

[0410] Ped(a) updates the hashing value for field [Phone] in SDB(a),generates and sends a packet 6 to Pr(a) containing the timestamp andfield value.

[0411] Pr(a) forwards the packet 7 to Pe(a) with the timestamp, fieldvalue, and synchronization S1.

[0412] Pe(a) reads the Synchronization definition of S1 in SDB(a). Fromthis, Pe(a) knows the destination database and the communication channelto use. Pe(a) forwards packet 7 to Pr(b). Note here that Pe(a) does notcheck the synchronization membership because the packet is an answer toa request.

[0413] Pr(b) receives packet 7. It compares the timestamp of field[Phone] in packet 7 with the value in SDB(b) and detects a modificationfor the field. It updates the value of timestamp in the SDB(b). In fact,the writing in SDB(b) does not occur immediately. The process waits foran acknowledgement (more precisely a return on function) confirming thatthe field value has been effectively written in database 472.

[0414] Pr(b) forwards packet 7 to Pe(b) 506.

[0415] Pe(b) forwards packet 7 to Prd(b) 508.

[0416] Prd(b) reads in the SDB(b) the record key corresponding to thesynchronization S1. With this key, Prd(b) selects the record in database472 and writes the value of the field [Phone] contained in packet 7 intothe Field [phone] of database 472. Finally, Prd(b) updates the hashingvalue of the Field[Phone] in SDB(b).

[0417] Change of query, suppression in destination source, user flag

[0418] This example illustrates the synchronization process following asynchronization query modification causing, according to the user flag,the suppression (or not) of the destination record.

[0419] In this example, two databases 470(A) and 472(B) are located inSan Francisco and Brussels respectively. Each database contains onetable.

[0420] Assume an initial situation described in FIG. 28 and assume toothat the synchronization definition S1 474 is changed:

[0421] From:

[0422] Source database 470(A) and destination database 472(B)

[0423] Synchronization query SQ1: select city=San Francisco

[0424] Source synchronization field mask: [Last Name, First Name, City,Phone]

[0425] Source Record key for selection of records in 470: Key=[LastName, First Name, City]

[0426] Destination synchronization field mask: [Last Name, First Name,City, Phone]

[0427] Record key for selection of records in 472: Key=[Last Name, FirstName, City]

[0428] Field matching between 470 and 472 for this example is shown inFIG. 29A.

[0429] To: the same definition except for the synchronization query:

[0430] Synchronization query SQ1: select [city]=Boston

[0431] Also, suppose here that no notification mechanism is available.

[0432] The synchronization process is as follows:

[0433] When Qes(a) starts and ends treatment of a query, it sends toQr(a) a start and end packet respectively. During the loop, Qes(a) doesnot select any record in the database A with the synchronization querySQ1 (select [city]=Boston) on database 470.

[0434] Qr(a) receives the end packet and starts a cleaning process forall synchronized fields for which it has not received information fromQes(a). In this case, a cleaning process is started for the fields inrecord A1.

[0435] The cleaning process removes from the SDB 482 all fields ofrecord A1 and asks Pe(a) to send a special packet “out ofsynchronization” to all synchronizations the record A1 belongs to. Inthis case a “out of synchronization” packet is sent from Pe(a) to Pr(b)indicating that the record A1 is out of S1.

[0436] Two cases must be considered here:

[0437] No field has been locally changed in database 472

[0438] Pr(b) receives the packet “out of synchronization” from Pe(a).Pr(b) removes the membership to S1 of all fields in record A1 in SDB(b).

[0439] Because S1 is the last (only) synchronization associated to thefields, Pr(b) sends a “delete” packet to Pe(b).

[0440] The “delete” packet passes through Pe(b) and arrives to Prd(b).Prd(b) checks in SDB the user flag and deletes the records in database472 because the user flag is “off”.

[0441] A field has been locally changed:

[0442] Suppose that the Field [Phone] in database 472 has been changedby a local access (from 1 415 842 5641 to 1 415 842 5965) after thechange of synchronization query, but before the “out of synchronization”packet arrives to Sb.

[0443] The local modification has initiated the following actions in Sb:

[0444] A local modification in Field[Phone] in database 472 is detectedby Qes(b) 510. Qes(b) updates the user flag in SDB 500 sends a packetcontaining timestamps for Field[Phone]

[0445] Pr(b) updates the timestamp in SDB 500.

[0446] The “out of synchronization” packet arrives after the localmodification.

[0447] Pr(b) receives the packet “out of synchronization” from Pe(a).Pr(b) removes the membership to S1 of all fields in record A1 in SDB(b).

[0448] The user flag set to “on” prevents Pr(b) from sending a deletepacket to Prd(b).

[0449] Merging:

[0450] This example illustrates the merging of fields coming from 2records in 2 databases into one destination record. Let's define 2synchronizations S1 474 and S2 512:

[0451] The synchronization S1 is defined by:

[0452] Source database 470(A) and destination database 472(B)

[0453] Synchronization query SQ1: select [city]=San Francisco

[0454] Source synchronization field mask: [Last Name, First Name, City,Phone]

[0455] Source Record key for selection of records in 470: Key=[LastName, First Name, City]

[0456] Destination synchronization field mask: [Last Name, First Name,City, Phone]

[0457] Record key for selection of records in 472: Key=[Last Name, FirstName, City]

[0458] Field matching between A and B for this example is shown in FIG.29B:

[0459] The synchronization S2 is defined by:

[0460] From:

[0461] Source database (C) 514 and destination database (B) 472

[0462] Synchronization query SQ2: select [city]=San Francisco

[0463] Source synchronization field mask: [Last Name, First Name, City,Address]

[0464] Source Record key for selection of records in 470: Key=[LastName, First Name, City]

[0465] Destination synchronization field mask: [Last Name, First Name,City, Address]

[0466] Record key for selection of records in 472: Key=[Last Name, FirstName, City]

[0467] Field matching between C and B for this example is shown in FIG.29C.

[0468] The detection method is the notification method for all thedatabases.

[0469] Initially, records 476(A1) and 516(B1) exist in database A and Brespectively (see FIG. 29D).

[0470] Suppose that a new record 518(C1) is created in C.

[0471] The synchronization steps are:

[0472] The database C notifies F(c) that a record (C1) has been added.F(c) reads the field values in database C.

[0473] F(c) reads all the synchronization queries defined in SDB(c) inorder to detect the synchronization(s) the record is belonging to. Inthis case, F(c) detects that the record C1 is belonging tosynchronization S2. Indeed, the record C1 responds to thesynchronization query {SQ2: select [city]=San Francisco}.

[0474] F(c) creates the corresponding fields and inserts in the SDB(c) ahashing function of the field values (a default value of timestamp isalso inserted at the creation time. This value is the “oldest timestamp”known by the synchronization system).

[0475] For the record C1, F(c) also inserts in SDB(c) a local identifier(IDlocal C1) and a global identifier (IDglobal C1). In a synchronizerthere is one unique IDlocal per record. For a record, its IDlocal isunique within a synchronizer while its IDglobal is unique within anetwork of synchronization.

[0476] F(c) applies the field mask and sends to Pr(c) a packet 1 withthe 4 field values, the 4 timestamps (current time) and their associatedsynchronizations.

[0477] Pr(c) compares for each field in packet 1 the timestamps in thepacket with the timestamps of the SDB. Pr(c) detects a difference forall fields. Pr(c) forwards the packet 1 to Pe(c).

[0478] Pe(c) reads the Synchronization definition of S1 in SDB(c). Fromthis, Pe(c) knows the destination database and the communication channelto use. Pe(c) sends to Pr(b) the packet 1.

[0479] When the packet is coming, the synchronizer tries to convert therecord IDglobal into an IDlocal via the information coming from theSDB(b). This conversion fails because the record does not exits yet. TheIDlocal is temporary set to 0.

[0480] Pr(b) receives packet 1. It compares the timestamp of fields inpacket 1 with the value in SDB(b) and detects a creation of fields.Pr(b) forwards the packet to Pe(b). and is waiting for a confirmationthat fields have been modified in database B before updating thetimestamp(s) in SDB(b).

[0481] Pe(b) forwards the packet to Prd(b)

[0482] Prd(b) builds a record C1′ from the packet 1. From the key recorddefinition of SQ2 in SDB(b) and the value of the key in record C1′,Prd(b) reads the record C1 in database B. Prd(b) compares the records C1and C1′ and detects that the field [Address] is not present in C1′.

[0483] Prd(b) inserts the value of the Field [Address] in C1. It checksif a record with the key value is present in SDB and sees that it is thecase. Prd(b) adds in SDB(b) the IDglobal C1 beside the already existingIDglobal Al. Prd(b) also adds the hashing function of thefield[address].

[0484] The key point here is the fact that only one record is maintainedin database B and SDB(b).

[0485] Pr(b), which was waiting, updates the timestamp forField[Address] in SDB(b).

[0486] It is understood that the functional aspects of the invention canbe implemented in hardwired circuitry, by programming a general purposeprocessor, or by any combination of hardware and software. For example,each of the functional aspects of the invention, such as Qr or Pd, maycorrespond to a sequence of instructions stored in a memory.

[0487] Although certain presently preferred embodiments and examples ofthe present invention have been specifically described herein, it willbe apparent to those skilled in the art to which the invention pertainsthat variations and modifications of the various embodiments andexamples shown and described herein may be made without departing fromthe spirit and scope of the invention.

What is claimed is:
 1. A data processing method for synchronizing datarecords of a source database and a destination database, the methodcomprising: defining a synchronization set for data records existing inthe source database; synchronizing the source database and thedestination database based on the synchronization set; and changing thedefinition of the synchronization set while synchronizing the sourcedatabase and the destination database.
 2. The data processing method ofclaim 1, further comprising: removing a data record from the destinationdatabase based on the step of changing the definition of thesynchronization set.
 3. The data processing method of claim 1, furthercomprising: deleting a data record belonging to the synchronization setfrom the source database; and removing the data record from thedestination database based on the deleting of the data record in thesource database.
 4. A data processing method for synchronizing datarecords of a plurality of databases, the method comprising: defining asynchronization set for data records existing in at least one of theplurality of databases; synchronizing the plurality of databases basedon the synchronization set; and changing the definition of thesynchronization set while synchronizing the plurality of databases. 5.The data processing method of claim 4, further comprising: removing adata record from at least one of the plurality of databased on the stepof changing the definition of the synchronization set.
 6. The dataprocessing method of claim 4, wherein the step of synchronizing theplurality of database includes merging data from at least two of theplurality of databases into a data record in one of the plurality ofdatabases.
 7. A data processing method for synchronizing data records ofa source database and a destination database, the method comprising:defining a synchronization for data records existing in the sourcedatabase; synchronizing the source database and the destination databasebased on the synchronization set; and scanning the data records in thesource database defined in the synchronization set.
 8. The dataprocessing method of claim 7, further comprising: receiving a pluralityof queries for information on data records defined in thesynchronization set; and Wherein the step of scanning includes combiningthe plurality of received queries before scanning the source database.9. The data processing method for synchronizing data records of a sourcedatabase and a destination database, the method comprising: definingsynchronizing set for data records existing in the source database;synchronizing the source database and the destination database based onthe synchronization set; and creating a notification packet for eachdata record defined in the synchronization set and modified in thesource database, the notification packet containing only an indicationof modification.
 10. A data processing method for synchronizing datarecords of a source database and a destination database, the methodcomprising: defining a synchronization set for data records existing inthe source database; synchronizing the source database and thedestination database based on the synchronization set; and creating auser flag each data record defined in the synchronization set andmodified in the destination database.
 11. A data processing method forsynchronizing data records of a source database and a destinationdatabase, the method comprising: defining a synchronization set for datarecords existing in the source database; hashing information on the datarecords defined by the synchronization set; sending the hashinginformation to the destination database; and synchronizing the sourcedatabase and the destination database based on the synchronization set.12. The data processing method of claim 11, further comprising:receiving a request for hashing information on at least on data recorddefined by the synchronization set at the source database from thedestination database; and sending the at least one data record to thedestination database.
 13. A data processing method for synchronizingdata records of a plurality of databases, the method comprising:defining synchronization links for a unidirectional synchronized datatransfer between any two of the plurality of databases; defining acombination of the synchronization links, for synchronizing a pluralityof databases; synchronizing the databases; and applying the databaserecord changes to all the linked databases.
 14. The data processingmethod of claim 13, further comprising: defining a query to identify asubset of the data records to synchronize from a source database foreach link.
 15. The data processing method of claim 13, furthercomprising: defining a subset of data records fields to synchronize froma source database for each link.
 16. The data processing method of claim13, further comprising: defining a matching between a source databasefield and destination database field for each link.
 17. The dataprocessing method of claim 13, further comprising: defining a merging ofa transferred data records in a destination database for each link. 18.The data processing method of claim 13, further comprising: defining amerging of data fields in destination database records for each link.19. The data processing method of claim 13, further comprising: defininga field transformation function for each transferred data field.
 20. Thedata processing method of claim 13, further comprising: encryptingsynchronized data fields at a source database and decrypting the datafields at a destination database.
 21. The data processing method ofclaim 13, further comprising: keeping synchronization information foreach data record of a synchronized database; comparing thesynchronization information between linked databases for each link; andsending the data records from a source database to a destinationdatabase, based on the comparison for each link.
 22. The data processingmethod of claim 21, further comprising: restarting the step of comparingwhen a synchronization link is broken and reestablished.
 23. The dataprocessing method of claim 21, further comprising: keeping a uniqueglobal ID for each synchronized data record.
 24. The data processingmethod of claim 21, further comprising: keeping a hashed value for eachdata field in the synchronized records; keeping a modification time foreach data field in the synchronized records; and updating themodification time when the hashed value is changed.
 25. The dataprocessing method of claim 21, further comprising: updating thesynchronization information by scanning the plurality of databases. 26.The data processing method of claim 25, further comprising: combining aquery of multiple synchronization links into one query for scanning. 27.The data processing method of claim 21, further comprising: updating thesynchronization information when receiving database changenotifications.
 28. The data processing method of claim 27, furthercomprising: activating the step of updating based on the status ofsynchronization information, while synchronizing.
 29. The dataprocessing method of claim 21, further comprising: keeping a user flagfor each field in the synchronized records.
 30. The data processingmethod of claim 13, further comprising: adding, removing or changing asynchronization link definition while synchronizing the plurality ofdatabases.
 31. The data processing method of claim 30, furthercomprising: adding records or fields into records in a destinationdatabase when a synchronization link is added or changed.
 32. The dataprocessing method of claim 30, further comprising: removing records orfields from a destination database when a synchronization link isremoved or changed.
 33. The data processing method of claim 32, furthercomprising: protecting a field from removal based on a status of a userflag.
 34. The data processing method of claim 13, further comprising:protecting the plurality of database coherence by transferring allmodified fields of a logical record together.
 35. The data processingmethod of claim 13, further comprising: accessing shared synchronizeddatabases while other processes or applications are modifying thesynchronized databases.
 36. The data processing method of claim 13,further comprising: keeping meta-data information for each synchronizeddatabase.
 37. The data processing method of claim 13, furthercomprising: merging all synchronization information for a synchronizeddatabase into an information database, the information beingsynchronization links information, record synchronization information,and meta-data information.
 38. The data processing method of claim 37,further comprising: synchronizing information database of linkedsynchronized databases; and creating synchronization links between theinformation databases.
 39. The data processing method of claim 13,further comprising: creating a destination database for eachsynchronization link; and changing a destination database structure foreach synchronization link.
 40. A computer system configured tosynchronize data records of a source database and a destinationdatabase, the system comprising: a processor; and a memory coupled tosaid processor; the memory having stored therein sequences ofinstructions which, when executed by said processor, cause saidprocessor to synchronize data records of the source database and thedestination database by causing the processor to perform the steps of:defining a synchronization set for data records existing in the sourcedatabase; synchronizing the source database and the destination databasebased on the synchronization set; and changing the definition of thesynchronization set while synchronizing the source database and thedestination database.
 41. An article of manufacture that includes amedium readable by a processor, the medium having stored thereon aplurality of sequences of instructions, said plurality of sequences ofinstructions including sequences of instructions which, when executed bya processor, cause said processor to perform the steps of: defining asynchronization set for data records existing in a source database;synchronizing the source database and a destination database based onthe synchronization set; and changing the definition of thesynchronization set while synchronizing the source database and thedestination database.